Symbolic Dynamic Programming for First-Order MDPs
نویسندگان
چکیده
We present a dynamic programming approach for the solution of first-order Markov decisions processes. This technique uses an MDP whose dynamics is represented in a variant of the situation calculus allowing for stochastic actions. It produces a logical description of the optimal value function and policy by constructing a set of first-order formulae that minimally partition state space according to distinctions made by the value function and policy. This is achieved through the use of an operation known as decision-theoretic regression. In effect, our algorithm performs value iteration without explicit enumeration of either the state or action spaces of the MDP. This allows problems involving relational fluents and quantification to be solved without requiring explicit state space enumeration or conversion to propositional form.
منابع مشابه
Symbolic Dynamic Programming for Discrete and Continuous State MDPs
Many real-world decision-theoretic planning problems can be naturally modeled with discrete and continuous state Markov decision processes (DC-MDPs). While previous work has addressed automated decision-theoretic planning for DCMDPs, optimal solutions have only been defined so far for limited settings, e.g., DC-MDPs having hyper-rectangular piecewise linear value functions. In this work, we ext...
متن کاملBounded Approximate Symbolic Dynamic Programming for Hybrid MDPs
Recent advances in symbolic dynamic programming (SDP) combined with the extended algebraic decision diagram (XADD) data structure have provided exact solutions for mixed discrete and continuous (hybrid) MDPs with piecewise linear dynamics and continuous actions. Since XADD-based exact solutions may grow intractably large for many problems, we propose a bounded error compression technique for XA...
متن کاملFaster Dynamic Programming for Markov Decision Processes
Markov decision processes (MDPs) are a general framework used in artificial intelligence (AI) to model decision theoretic planning problems. Solving real world MDPs has been a major and challenging research topic in the AI literature, since classical dynamic programming algorithms converge slowly. We discuss two approaches in expediting dynamic programming. The first approach combines heuristic...
متن کاملEfficient Symbolic Reasoning for First-Order MDPs
We propose an algorithm, referred to as ALLTHETA, for performing efficient domain-independent symbolic reasoning in a planning system FLUCAP 1.1 that solves first-order MDPs. The computation is done avoiding vicious state and action grounding.
متن کاملSymbolic Dynamic Programming
Decision-theoretic planning aims at constructing a policy for acting in an uncertain environment that maximizes an agent’s expected utility along a sequence of steps that solve a goal. For this task, Markov decision processes (MDPs) have become the standard model. However, classical dynamic programming algorithms for solving MDPs require explicit state and action enumeration, which is often imp...
متن کاملSay “No” to Grounding: An Inference Algorithm for First-Order MDPs
We propose an algorithm, referred to as ALLTHETA, for performing efficient domain-independent symbolic reasoning in a planning system FLUCAP that solves first-order MDPs. The computation is done avoiding vicious grounding.
متن کامل